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An experimental desiign vas^ employed tp ass^s the 
impact of standardized tests on *7 sec6 nd grade ^rish teach)^rs« 
judgments of their 1#566 students. Teachers redfeiving test results 
showed greater shifts ^in theijc ratings of students than teachers, not 
receiving test results/ Bdmeverr these shifts tended to be in the 
po^\ive direction and ^or only a small number of students. Besults 
indicated that standardized test results di^d alter second grade, 
tei^chfers^^ expectations for their students in- a small petcentage of 
cases (1W oi; less) . This study also indicated that teachers trended 
• to raise> but not lower ^ their ratings of students' perfoi^mattce as a 
result of receiving test results. For at least 9t% of ^the /cases, 
ho¥*Ter, the information provide^ by standardiz€| test results seemed 
to either corroborate the teachers' existing expectations or to be 
too vealc to al^er their jBijisting expectations. Since this study took* 
place in Ireland, the relef.ance of these filndings for the United 
States' was also discussed. The authors ci'ted reports indicating that 
jrish/^ieachers liave m;ore favorable attitudes toward testing than 
American teachers. Therefore, they concluded that the influence of 
test resu4.ts' in American teachers' expectations would be ^ven less 
than theinfluence on Irish teachers tescribed in this study. 
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^ , Cyne area of concern in the c\irrent controversy over the effects of ' standard- 

ized testing is the impact of standardized test results on teachers • /eaqpectationa 

' ' * ' . ^ . "I J '\ 

for, their students. ThfiLJpecent resurgence of interest in this issu^ was provided 

I * ^ * * ' I I • 

by Rosenthal and Jacobson (1963) , who argue^ that providing teacheps with contrived 

/ 7 

and inflated test results inSrease^ students' subsequent t^st p6r|orq(ianqe. More 

* "* ^ '' / ' 

precisely, the reasoning went as fo^low^:' (1) teachers hav^ expefct^ioAs for 

their students, (2) teachers attend to the test results in judgirig i^upils, {ti by 

\ . • . • ; ^ 

pft6viding' contrived test results to teachers, their existing ejcpectationa for st^u- 
dents c^n be altered, (4) the change in' expectations will be tr^ij^lated into \ 
changes in the teacher-student interaction pro'^ess, (5) these changes In ^inter- 
actions will reflect the new expectations for pupils .which were \>a&%d oi their con- 
triv€4 test scores, and (6) the pupils will conform to tbs xvpw beh4viorl e:q)ected 



of them. 



^ Rosenthal and Jacotoson's results were widely publicized and # ^ tio | this dasf^ are 
acfcepted by many people in education despite -the fact that the sttkiy Was Istrongly 
criticized for its methodological and analytical flaws (Eldshoff anb dnbwl I97;i; . 



Snow, 1969? ThomdiJce, 1963). The results of many replication attei 
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Ass6ciation, Uew York, April,. 1977. Work on this study was supported furHs from 
. the' Carnegie Corpcfration of New York, the Russell Sa^e Foundation^ thelspenwr ^ 
Poundation, and the National Institute for Education, The stat^ents tirade aiVd the' 
^ -views, expressed are solely the responsibility of the authors. 



lERiC 



2 > 



♦'PERM^SS^aij TO REPlpOOUCjE THIS 
MATERIAL j AaS' BEEN GRANTED BY 



TO THE EDUCATIONAL' RESOURCES 
INFORMATION CENTER (EWIC) AND 
, 'users OF'tHE ERIC SYSTEN|" 



regulkx* classroom' setting failed' to ccrrcborate the Rosenthar and Jacobson (1968) 
findings (cf. Fleming "and Anttonen, 1971; Jcse and Cody, 1971; Hendels and Flanders, 
1973; Wilkitis and Clock, 1973). Thus,, studies att^.pting to artifici^Uy^induce, 
raised expectations in regular classroom teachsrs by using ^contrived test score data 
have generally failed to do so. 

.* There are many possible explanations 'for the failure of indiaced expectancy 
Studies to find strong effects. One is t^iat teachers have well-fbrm^d, relatively 
stable expectation? for their students based on clcissroom observation and past teach 
ers' comments (Airasiaia", Kellaghan and MadaUs, 19J5; Past, 1970; Uillis, 1972) • A , 
single,, additional piece of information, such as a standardized test score, may in 
fxict have very little influence on a teacher's pre-existing .expectation for a stu- 
dent. This may be particularly true when the test results are false, since the re- 
sults may be too discrepant from the teacher's existing expectations for the 
student to^be accepted' (Fleming and Anttoneo, 1971; Pedulla, 1976) . 

Another possible explanation for the inability of induced expectancy studies to 
find effects may be due to the fact that only raised e3?>ectations were induced. It^ 
seems reasonable to assume that, if standardized test results affect expectations, 
they may lower, as well as raise, some expectations. . By examining onl^^aised ex- 
pefctations, researchers are looking at only^alf of the phenomenon and thus may de- 
crease their chances of ^ finding effects* 

; A third possible explanation for the lack of significant results in iriduced 
expectancy studies may be related to 'the pervasiveness of standardized testing in 
the United Srates where* these induced expectancy studies have taken place. Assess- 
ing the effect of providing, teachers with contrived test results becomes confounded 
with any, previous "real" test results in the students' permanent record and with 
teachers' prior experie^^ces. with ^nd attitudes toward standardized tests. The re- 
searchers cannot disentangle the effect of previous exposure to standardized tests 
from the ef fects^the/' are investigating. .Thus, the treatment is confounded in 



.dnaoc«> e^ect,„ci- .tudies,' and this o..foa,aed trei^nt n,,y „Ute .,.tn.t fi^- 
ing the anticipated results. 

The purpose of the' present study was to exonui the effect of «,tuU ,^4>rd- 
i>ed test result, on teachers- expectation for iudent. e^ evidenced bs-, changes 
> teachers •'-rati^ts of each stadenfs a=adUoy4billty.. Thu. the present st»Jy . 
ha, eliMnate-d of the. probZe,s which.n,ay derate e^ai-u-t finding effects on 

teadhers' e:<pectatio„s *.e to standardized t^^t results, students' actual test ' 
• scores (net artificial ones, „ere' reported Ack to the teachers. In this «y,' the ■ 
artificial nature of the test results use/ in pre.liu, expectancy studies was 
eliminated and the influence of standard/zed test'result, on lowering, a, ^U-as 
raisin,.^ teachers' expectations could detennihed.' The confoundim, of p«vious ,' 
experiences with and attitudes toward/standardized testing .ith the tireat«ent ,a. 
«ini.Uzed,by.conducting the study in/a settir^ in -,i* teachers and student, had ^. 
virtually no prior experience with /tandardized testing. . 

Meth^ • . ' . / 

Subjects • / ■ / ' ' 

A sample of 47 second grade /teachers/ teaching 1566 students '^was randomly . ' • 
. selected from a larger sample ofLcond ^rade teachers and students participating ' " 
in a societal experiment design! to investigate a wide raAge of issues associated 
with the effects of standardized testinc " 
conducted in the Republic of Ire|lai.d witi a nationwideLandom sample of 230 IrW' 
schools; over 1500 teachers", and over 40^000 pupils atlll grade levels/ Since 
teachers and student, ia the Republic' of Iceland had viltually no e,^rience with ' 
standardized testing prior to the coAtrolfed. introauctiol of su<,„ t.sts in this - 
societal experiment, it was possible. to e:;^ne the effl/of tests on teadu^rs . 
with few confounding conditions. ' * ^ 



. ^ The second grade was selected for study because most previous expectancy' 
studies have concentrated on the lower erases! It^ ;as\at.the lower grades ttat 
Rosenthal and Jacobson found their strongest effects. Further, it can be argued 
convincingly that it is at the loyer gries that teachers' expectations for p'upils 
^«.uld be most malleable; the'ticne a pupil is in the higher grades, the informal 
con^unication network operating £n .o'st schools, in conjunction wi^th the p^i^s 

^ history Of acadenic^ performance,, may have essentially -pegged" the -pupil with a 
fa>rly rigid -expectation. The effect of standardised \.st .results on these expec 
tati<ins wculd, therefore, probably be slight. Since gxT^de two.is the\owest ' ^ 
grade ganpied in the larger^ study, it vas selected for ' investigation. 



Procedure 



. The standardized nonnlref erenced tests used in" th'is study were the Drumcondra 
2251 series nathematics conputation and English reading subtests^. These tfests 'were 
built specifically for grade two Irish children and nortned using a national random 
sanple of second grade Irish students. In most respects', the Drt^cond^ Tests were 
very similar to their United States 'counterparts / 

This study utilized ^ classical pretest'posttest experimental design with the 
manipulated variable fce.ng whether ox not teachers' received nor»-referenced ' 
standardized achievement test results for the students in their class. In the ' 
larger experiment schools w^re randomly assigned to treatment groups. One treatment 
group -did not administer .any. standardized tests. A; second treatnnent 'group ad«ihi:s- 
tered the staiidardized, <.^sts but did not receive the results from thL'terts. A 
third treatment groyp administered and received the results from the Standardized 
tests in the /orm of raw vsc ores and percentile and standard scores based on the 
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national nom group. There were 11 teechors and 411 students in*t!he no testing 
group, 8 teachers and 269 students in/the testing but no results group, and 27 
teachers and ^78 student's in the testing v^ith results group. 

zed tests of English and mathematics were administered in Nbvem-^ 
prior to the test administration, teachers in all three treatment 
groups provided ratings of- each stude'nt's ^position relative to the other pt^ils ,in 
, the cla^s in En<3flish reading and mathemati/s cojtqputatioh. ' Ratings were pn a five- 
point scale v;ith '1' signifying that the student v/as in the bottom fifth of the 
class and '5' signifying that the student was in the top fifth of the class. .In 
late January or early February, test results, in the -form of raw scores,* percen- 
tiles and standard scores, were rejported back to those teachers who received re- 
feults. In iMay, ^11 teachers rerated each of their students in mathematics and 
^^glisl). ' ' ^ 

Changes in the pupil tatings wei-e examined for trie three treatment groups to 
determine' v;hether the availability of test information- resulted in greater rating 
changes. The nagnitude and direction of the rating changes were examined separ- 
ately- to deterciine not only whether test results affected teachers' ratings of -their 

pupils, but also whether test results fended to change ratings in a particular 

i 

direction. • . . » ^ , 

.To this en4, for each student, three indices were calculatecL for each of the 
two ratine^ areas . The first index measured the laagnitude of the chcuige in a ptjqpil'^s 
rating pair by taking the absolute value of the difference in that pair.' ?or' ex-^ 
ample, if the teaoher rated a student to be in the top fifth of the cXass in mathe- 
matics in tHe fal^ (a rating of .'5') and tlie middle fifth' of. the class in the 
sprirtg^ {a rating of 'B'), the value of the magnitude neasuro^for mathematics for 
that student would be 2. . ' t . % , . 



• - 
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The remaining indices dealt with the direction of rating shifts'. One index 
indicated positive rating shifts. If rhe pupil's final rating was higher thcui his 

initial- rating, thi^ index was coded ^1'; otherwise this index was coded 'O*. 

• -J ^ 

An index which indicated^ negative ?atings shifts was conputed in a siioilar fashion, 
.-i.e^. , if a pupil's final rating was lo^'er tKan his initial rating^ this index was 
coded '1'; it was coded '0' otherwise. It shSuid be noted that the means for^the 
variafctles measuring direction of rating shifts are directly interpretable as the 
proportion of ratings shifting in^ that direction. . ^ 

In sum, each student had a total of six indices, three for mathematics and 
em analogcas three for reading; (i) th^ magnitude of the change from initial to 
final ratin^> (2) whether the final ^^ting was higher than the ini1:ial rating; and 
(3) whether the final rating was lower than the initial rating. Each of these six 
indices v/as used as a dependent vairiable in a univariate , one-factor analysis of 
v£u:iance. In each analysis, the independent' variable was treatment group member*" 
sriip, i.e., no testing, testing but no results, or testing with results. PostL hoc 
con5>arisons' of means were conducted when statistically significant differences 

were found. 

J 

A • ^ . ' . . ' 

^ . \ * . Results 



Similar ifesults v/ere obtained for tlie corresponding rating-change measures 
'for English and mathematics. .This similarity was partially a function of the high 
cbrrelati9ns between magnitude, positit^e-or negative rating shift measures in ^ 
mathematics with the corresponding measure in English. ,^ The correlation matrix Is ^ 
presented in Table 1 and shows that these correlation* were approximately .^0, ThuSr 
• jtrtere was a "h^lo** effect in the ratings, i.e.^ a change in the English ratings for 
a student tended to be accompanied by a similar change in the mathematics ratings. 



1- 

Insert Table 1 here * 
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Tbe positive correlations betv;een the magnitude vaarij^les arid the direction 

variables' in the same svii^Ject area are partially a function of the ir^ner in which ^ 

* • * * 

^ these variables were constructed. Uon-zejjo chs^jife, in teems oi?^ magnitude oust # by 

definitiori, be related to positive and nec/ative rating shifts. Similarly, the 
negative correlations between variables r.easuring pisitiv^/ rating shifts and vari- 
ables negative rating shifts in the same subject area are a fynction of the manner 
in whiAi the variables were constructed, since ratings can shift in one and only one 
direction. The correlations are of a high ; enough magnitude to indicate that the 
analysis of variance results are dependont and must ^e interpreted in light of this 

\ ' ' •■ ' . 

/ dependency. . . \ * - _ 

/> ^The means and standard deviations for tlie variables are presented in Table 2. 

The* analysis of variance results for all six dependent measures are presented in 
Table 3. The analysis of variance results for both- the EUi^lish* and mathematics 
magnitude of rating shift variables. Were statistically significant^ (£ <.05) . Post, 
hoc comparison's of means, using the* Tu^iey test fojj honestly significant differences 
(Winer; 1971) , showed that the group which received test results exhibited signi- 
ficantly greater magnitude of ratihg change thane (1) either of the other two 
groups for the English ratings and (2) thfe'no testing group for the .mathematics 
ratings.. , j ^ 



Insert Tables 2 and 3 here ' ^ 

• (. 



'statistically sicjnificant differences were also found for both the English and 
jtvathenatics variables measuring positive rating shifts (cf. Table, 3)^ That is, 
for both English and mathematics^ th^re v;as a difference between the three treatment 
groups in the proportion of" ratings which were raised. Post hoc comparisons of 
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of means showed that the group which received -^^st results^raisjsd ,inore of their. . 
^ratings in English than the g^roup which tested biit received no results. In mathe- 
matics, the group which received test results rai^^ laore of their rating^, than the 
group which^did not test/. ^ , 

The means for the positive^ rating change measures are directly interpr^table 
as the pfoporl^ion of ratings shif ting .lipv/ard. Keeping this interpretatipn in mind, 
one can see fron Table 2 that between 13% and 23% of all the ratings shifted ypward. 
By using the mean from either the no tWsting o? the testing but no results gTot^s 
as a baseline for rating sfhifts due to factors, other than receiving standardized ' 
test resul^ts, one c/an obtain an indication of the proportion of shifts directly ett- 
tributable to receiving standardized test results. » Whether to use the mean oC - 
•the no testing or testing but rt^ resuJ^t^ groups was established by which cfrie wa^ 
Agnificantly different from-^e mean of the'groi:^) which received results. ' This* 
approach yields the most liberal estimate of the proportion of rating changes 
directly attributable to receiving standardized test results. .Using Jthe appropriate 
baseline mean, one finds that 7% of the English ratings and 1Q% of -the'matheinatics 
ratings shifted upward solely as a consequence of receiving standardized test re* 
suits. 

\ 

No signif iycant differences aiqpng treatment groups were found for tha negatiye 
rating shift measures. The means in Table 2 do show,, howev^, that rou^ly 20% 
of the ratings in all groups shifted downward. Since roughly 20% of the ratings 
also shifted upward, approximately, 60% of all ratlfngs did not chamge at all. 

Thus^ the results from this study shov; a "halo" effect in the ratings* They 

^ * ' * 

also show that less than half of the teachers' ratings of pupils changed at all over 
the course of the school year. Qf^ those 40% or so of the ratings which did change. 



Qply a small percentage of the ^changes (7%-iaPa) could be ^tt^ibu^ed directly to 

being in the group which- received «Gt results. Finally^ the results fron this 

study indicate that receiving stand^rdi::ed test results tended to»raise, but not -^^ 
loi^rer,^ tead^ers* ratings of their students. 

. , J , , » Discussion . ^ ' < 

Teachers have velry st^rong and stable- expectations for their students early 

in the school *year. Over half of theie expectations are so fixed that they do 

i^ot change at all over ^e course of the school year. Evidence- of this rigidity 

iiv expectations was provided by the fact that 60% of the initial-final rating pairs 

\ 

for Ijoth English and mathematics on a five-point sca^^e. were identical.' This find-y^ * 
ing corroborates the results of other studies which indicate that teachers font^ 
expectations early in the school year based on maijy types of cues fron or informa-' 
* tion about the student and that the3e„ixiltial Expectations are lasting (Fleming and 
Anttonen, 1971; Pedulla, 19"Z6;'Rist, l^yo, Wilkins and GlQpk/ 1973) . it should be 
noted that -these expectations v/ere formed independent of any, stsmdardiz^d test 
suits. ' • • /. • 

The changes that do occur, in teacher expectations ^ as evidenced' by their 
ratings of students , .showed^ a "halo'' effect. Teac±ers do not seem to discriminate 
^ between English and mathematics ^ery mucli. if they rcd.se thlir* estini^es of a 
^ student in English, they tend to raise th^ir estimate in mathematics also'.'. It 
seen?s that teacher expectations are formed and altejs^at a level more global th^ 



the sL:)ecific subject area level. Students are viewed as "smart" or "slow" in 

.general, not "smart" in some areas and "slow" in others. 

^ . i 

How., then, do standardized test results fit l^to the overall expectancQf pic- 

tore? The results- from this study indicate that standardized tejst results do 

alter grade 2 teache^' expectations for students in a small percentage 



of cases (10% or less). However, ;co^rairy to the det'rimental effect that some 
critics of standardized testing have clair-edv this itudy indicated that teachers 
tended to raise, but not- lower, their ratings of students' performance Js. a resttlt 
of receiving standardized test results . Howev«:, it must be emphasized that in the 
vast majority of case?, standardized test results had no influence on teachers' " 
ratings. Thus, , the new piece of. infomation which staodardized test results pro- . 
vide to teachers seems to either corroborate their existing expectations or be.tdo 
weak to alter existing expectaUons for at least 90% of the cases. " , 

Since this stu^ took place in the •. Republic of Ireland, the relevance of the^ 
findings for the United States nust be addressed. One coirparison'ibiat can'^be ^lad^ 
is Betv-een Jtaeriban and Irish teachers' attitudes and opinions toward standardized 
tests. The items for this comparison were employed, in a study conducted in the* 
United States over t^n, years ago (Brin, Glass and Goldberg;. 1965) ^nd again with • 
Irish teachers (Airasian, Kellaghan^nd Madaus, 1975). a coiroarison of the results 
frpm the Brim, et al. study to the Irish replication found that Irish' teachers had ■ 
more favorable attitudes, toward startdardi;:ed tests than tiieir Anerijsan counterparts 
(Airasian, Kellaghan and Madaus , 1975). 



, - From this fmciing it can *be aifgued that standardized tests Would have morja in- 
fluence on Irish teachers than on Airerican teachers. T& this is th^* cas^, the in- 
fluence of standardized test results on" American teadiers Ve35>ectation3 would be 
slight indeed. This sntall influence nay 'account for the inability of induced ex- 
pectabion studies to find effects, since in fact the effects nay not be there to ' 
find. I / " * • ' ' ' 

This study ■ examined' 'the Expectancy phenomenon •in the grossest pf ways, i^e.,' 
at the global treatment l^evel. As has been suggested' (Brophy and Good, 1974), a 
clearer_^derstan<3ing 6f teachers' expectations can be obtained by exaiaining' smaller. 



* . . i' , ■ ■ ,' ' ' ' • * " - ■ 

selected subgroiipi ,^ such as teacher^? ^wit]ii^, t^e'^er.t or' styadenl^ within teachers ' " 
witJiin tfeatment. Although initial- attenvpts it;this apEf:6ach witli W data for , 
this study failed to produce miicfa by uAy 'ofNidentifying particular types of teachers 
of students for wton standardizeV'^ge^. r^^l^k 'We ilBsj/^nf luential •<Pedulla, -l^Te), 
further research in this vein seems necessaty.. 'v • • 

In sum, the resplts from thirf s'tudy indicate that standardized test' results - 

'/■'.. - > *■ • * . K ■ 1 

affe|:t teachers' ratings for a small number of 'strfdents.., and the influence of~4iie' 
i^esults is to -raise, .but not lower, ratings. Thus, the criticism that standd^dized ' 
tests work to the detriment of sone students by categorizing , them ^s poor achievers^ 
in the teachers' eyes not onlj^ was not supported by this, study, but the reverse ' . • 
occuri:ence was fcnind. . . ^ ^ \ 
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Correlations Among "the Dependent. Variables 
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2. Magnitjude of change in mathematics 
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Means and Stemdard Deviations* by Treatment Group for 
AH* Dependent Measure's 
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V«feLable , ^ ' 




Treatment Group 






No 
Testing 


' Testing - 
* "i^o Results 


Testing 
. ^ and Resqits _ 


Magnitude of change 
in English ratings 


.•39 


> • •••.39 . '• 
(.54) 


. .49 ' . 
(.61)' 


Magnitude of change 
in natliematics ratings 


' .38 
(.54)^ 


.45 • 
(.55) 


. .55 

(.66) 


Positive shifts in 
English ratings 


.20 
' (.40) . ^ 


(.37) 


~ . .23 • 
(.42) 


Positive ^iits in 
mathematics ratings 


.13 • 
(.34) 


.13 
(.39) 


.23 V. 
* (.42) 


Negative shifts in- 
English ratings 


.16 
(.37) 


(.40) 


.20 
(.40) • 


Negative shifts in^ 
iiatheinatics ratings 


.22" 
(.4^) 


.22 

(.42) ^ 


.23 
(•42) , , 



Standard deviations - cure presented in parenthese|S. 




Witiiih groups 



Beftween ^oups 
Within groups 



stween groups 
llithin groups 



^tween groups 
Within groups* 



Between groups*' 
Within groups 



\ Between groups 
^ Within groups 



Magnitude of C3i^ge' in EiCgUsh Rating* 
'^-^^ 2.37 - 7.04. .001 




, ■ 

Magnitude, of Change . in, Mathematics Ratfng* 



. V J 2 ,7.29 ' 3 64 
1463 571.12 . p.'ag 



9.33 



001 



: ' Positive Shifts in English Ratings 



positive shifts in 'Mathematics Ratirigs 

7.33. ., .001 




/ 



16 



ERIC 



